Equilibrium Propagation (EP) is a biologically inspired learning algorithm for convergent recurrent neural networks, i.e. RNNs that are fed by a static input x and settle to a steady state. Training convergent RNNs consists in adjusting the weights until the steady state of output neurons coincides with a target y. Convergent RNNs can also be trained with the more conventional Backpropagation Through Time (BPTT) algorithm. In its original formulation EP was described in the case of real-time neuronal dynamics, which is computationally costly.

The authors first introduce a discrete-time version of equilibrium propagation (EP). They show the equivalence of EP with backpropagation through time (BPTT). They also apply it to a CNN (first time). They show step-by-step equality under certain conditions. All reviewers agree that the results are original, the quality and clarity of the paper is high, and the results are very significant for the NeurIPS community, in particular to researchers interested in biologically plausible replacements of backpropagation.

The manuscript describes a discrete time reduction of equilibrium prop (EP) which enables the authors to compare the algorithms gradient estimates and performance directly to BPTT. Moreover, the associated reduction in compute cost also enables them to train the first CNN (that I know of) using EP. While EP approximates BP in feedforward networks, it uses neuronal activity of an RNN in equilibrium to propagate target information or error feedback to perform credit assignment. While this work may be less interesting for DL practitioners because it is still more costly than backprop (BP), it is one of the contenders for bio-plausible backprop which is discussed in the literature. In that regard the present work contributes to this discussion meaningfully.

Equilibrium Propagation (EP) is a biologically inspired learning algorithm for convergent recurrent neural networks, i.e. RNNs that are fed by a static input x and settle to a steady state. Training convergent RNNs consists in adjusting the weights until the steady state of output neurons coincides with a target y. Convergent RNNs can also be trained with the more conventional Backpropagation Through Time (BPTT) algorithm. In its original formulation EP was described in the case of real-time neuronal dynamics, which is computationally costly.

Equilibrium Propagation (EP) is a biologically inspired learning algorithm for convergent recurrent neural networks, i.e. RNNs that are fed by a static input x and settle to a steady state. Training convergent RNNs consists in adjusting the weights until the steady state of output neurons coincides with a target y. Convergent RNNs can also be trained with the more conventional Backpropagation Through Time (BPTT) algorithm. In its original formulation EP was described in the case of real-time neuronal dynamics, which is computationally costly.